Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding

Abstractive community detection is an important spoken language understanding task, whose goal is to group utterances in a conversation according to whether they can be jointly summarized by a common abstractive sentence. This paper provides a novel approach to this task. We first introduce a neural contextual utterance encoder featuring three types of self-attention mechanisms. We then train it using the siamese and triplet energy-based meta-architectures. Experiments on the AMI corpus show that our system outperforms multiple energy-based and non-energy based baselines from the state-of-the-art. Code and data are publicly available.Comment: Update baseline

Speaker-change Aware CRF for Dialogue Act Classification

Recent work in Dialogue Act (DA) classification approaches the task as a sequence labeling problem, using neural network models coupled with a Conditional Random Field (CRF) as the last layer. CRF models the conditional probability of the target DA label sequence given the input utterance sequence. However, the task involves another important input sequence, that of speakers, which is ignored by previous work. To address this limitation, this paper proposes a simple modification of the CRF layer that takes speaker-change into account. Experiments on the SwDA corpus show that our modified CRF layer outperforms the original one, with very wide margins for some DA labels. Further, visualizations demonstrate that our CRF layer can learn meaningful, sophisticated transition patterns between DA label pairs conditioned on speaker-change in an end-to-end way. Code is publicly available

Meeting Intents Detection Based on Ontology for Automatic Email Answering

National audienceAutomatic email answering is a difficult AI problem that combines classification, natural language understanding and text generation techniques. We present an original approach and a tool based on an ontology to automatically reply to meeting emails. We constructed the ontology from a French corpus of 1150 emails in which the concepts represent detailed meeting intents (proposing a meeting, cancelling a meeting, rescheduling a meeting) and different answer templates. Each intent concept is a semantic rule formalized according to the FrameNet methodology. These rules are used to detect intents in emails and also to extract relevant information (such as date, time or person) used for generating replies. The main advantage of our approach is the generation of more precise answers than those proposed by other approaches. We tested the intent detection step on a set of 297 emails and compared it with different supervised machine learning algorithms. Obtained results are encouraging, with an accuracy 20% higher than results obtained with other algorithms. Mots-clés : Ontology engineering, knowledge acquisition from text, knowledge-based recommendation systems

Enhancing Services Selection by Using Non-Functional Properties within BPMN in SOA Context

Part 11: Services IInternational audienceIn the Future Internet vision, multiple services coming from heterogeneous organizations have to collaborate together in order to achieve the customers’ demands from both functional and non-functional point of view. Hence, it is necessary, within an organization, to put in place an interoperable approach that ensures the best functioning control and selection of services. This paper presents a comprehensive framework for representing the customer Non-Functional Properties (NFP) within a collaborative Business Process Management (BPM) and the contribution of the Service Oriented Architecture Governance (SOA Governance) to give customers a better selection of services that best suits their business NFPs requirements among all Web Services candidates provided by functional matching

Data Programming for Learning Discourse Structure

International audienceThis paper investigates the advantages and limits of data programming for the task of learning discourse structure. The data programming paradigm implemented in the Snorkel framework allows a user to label training data using expert-composed heuristics, which are then transformed via the "generative step" into probability distributions of the class labels given the training candidates. These results are later generalized using a discrimina-tive model. Snorkel's attractive promise to create a large amount of annotated data from a smaller set of training data by unifying the output of a set of heuristics has yet to be used for computationally difficult tasks, such as that of discourse attachment, in which one must decide where a given discourse unit attaches to other units in a text in order to form a coherent discourse structure. Although approaching this problem using Snorkel requires significant modifications to the structure of the heuristics, we show that weak supervision methods can be more than competitive with classical supervised learning approaches to the attachment problem

A model-driven approach for collaborative service-oriented architecture design

In a collaborative context, the integration of industrial partners deeply depends on the ability to use a collaborative architecture to interact efficiently. In this paper, we propose to tackle this point according to the fact that partners of the collaboration respect the Service-Oriented Architecture (SOA) paradigm. We propose to design such a collaborative architecture according to Model-Driven Architecture (MDA) principles. We aim at using business models to design a logical model of a solution (logical architecture) as a principal step to reach the final collaborative solution. This paper presents the theoretical aspects of this subject and the dedicated transformation rules.Process Modelling BPMN Information system Transformation rule MDA SOA

Learning Multi-party Discourse Structure Using Weak Supervision

International audienceDiscourse structures provide a way to extract deep semantic information from text, e.g., about relations conveying causal and temporal information and topical organization, which can be gainfully employed in NLP tasks such as summarization, document classification, sentiment analysis. But the task of automatically learning discourse structures is difficult: the relations that make up the structures are very sparse relative to the number of possible semantic connections that could be made between any two segments within a text; furthermore, the existence of a relation between two segments depends not only on “local” features of the segments, but also on “global” contextual information, including which relations have already been instantiated in the text and where. It is natural to try to leverage the power of deep learning methods to learn the complex representations discourse structures require. However, deep learning methods demand a large amount of labeled data, which becomes prohibitively expensive in the case of expertly-annotated discourse corpora. One recent advance in the resolution of this “training data bottleneck”, data programming, allows for the implementation of expert knowledge in weak supervision system for data labeling. In this article, we present the results of our application of the data programming paradigm to the problem of discourse structure learning for multi-party dialogues

Weak Supervision for Learning Discourse Structure

International audienceThis paper provides a detailed comparison of a data programming approach with (i) off-the-shelf, state-of-the-art deep learning architectures that optimize their representations (BERT) and (ii) handcrafted-feature approaches previously used in the discourse analysis literature. We compare these approaches on the task of learning discourse structure for multi-party dialogue. The data programming paradigm offered by the Snorkel framework allows a user to label training data using expert-composed heuristics, which are then transformed via the "generative step" into probability distributions of the class labels given the data. We show that on our task the generative model outperforms both deep learning architectures as well as more traditional ML approaches when learning discourse structure-it even outperforms the combination of deep learning methods and hand-crafted features. We also implement several strategies for "decoding" our generative model output in order to improve our results. We conclude that weak supervision methods hold great promise as a means for creating and improving data sets for discourse structure

Char+CV-CTC: Combining Graphemes and Consonant/Vowel Units for CTC-Based ASR Using Multitask Learning

International audiencePrevious work has shown that end-to-end neural-based speech recognition systems can be improved by adding auxiliary tasks at intermediate layers. In this paper, we report multitask learning (MTL) experiments in the context of connectionist temporal classification (CTC) based speech recognition at character level. We compare several MTL architectures that jointly learn to predict characters (sometimes called graphemes) and consonant/vowel (CV) binary labels. The best approach, which we call Char+CV-CTC, adds up the character and CV logits to obtain the final character predictions. The idea is to put more weight on the vowel (consonant) characters when the vowel (consonant) symbol ‘V’ (‘C’) is predicted in the auxiliary-task branch of the network. Experiments were carried out on the Wall Street Journal (WSJ) corpus. Char+CV-CTC achieved the best ASR results with a 2.2% Character Error Rate and a 6.1% Word Error Rate (WER) on the Eval92 evaluation subset. This model outperformed its monotask model counterpart by 0.7% absolute in WER and also achieved almost the same performance of 6.0% as a strong baseline phone-based Time Delay Neural Network (“TDNN-Phone+TR2”) model